Multimodal Communication in Inhabited Virtual Environments

This paper reports on ongoing research of various aspects of multi-modal interactions in a Virtual Reality environment in which visitors can interact in diverse ways with agents and objects in the environment. The virtual environment is a copy of a theatre and is inhabited by both embodied and non-embodied agents that can assist the visitor: providing information about the performances, selling tickets, or helping the visitor navigate through the building. Interactions in this environment are typically multimodal. Several agents are capable of natural language interactions with the user. However, because the interactions take place in a virtual reality environment, a lot of information can also be presented visually.